Smoothing the power curve of LLM training and inference
Which kernel- and scheduler-level techniques reduce power ripples during LLM training and inference, and at what cost in energy and throughput?
GPU power draw during LLM training and inference is bursty: compute-bound prefill spikes, memory-bound decode idles far below the power ceiling, and synchronised training steps add further lumps. Across 89 sources (2011-2026) the evidence shows GPU frequency and power-capping settings have pronounced energy sweet spots (routinely 15-35% energy saved for single-digit throughput loss) that apply only to compute-bound phases; the newest work shows the power ramp between scheduling steps, not the peak, is the controllable quantity. Confidence is moderate: device-level trade-offs are replicated across many groups, while the ripple-smoothing literature is young (mostly 2025-2026 preprints) and largely single-node or emulation-scale.
Updated 6 Sept 202689 sources2011–2026Deep25 min read
GPU power capping · DVFS · power smoothing · energy-throughput tradeoffs · LLM inference energy · LLM training energy · prefill decode · tokens per watt